Supplemental Information : “ Robust exponential memory in Hopfield networks ”

For an integer r ≥ 0, we say that state x∗ is r-stable if it is an attractor for all states with Hamming distance at most r from x∗. Thus, if a state x∗ is r-stably stored, the network is guaranteed to converge to x∗ when exposed to any corrupted version not more than r bit flips away. For positive integers k and r, is there a Hopfield network on n = ( 2k 2 ) nodes storing all k-cliques r-stably? We necessarily have r ≤ bk/2c, since 2(bk/2c+ 1) is greater than or equal to the Hamming distance between two kcliques that share a (k − 1)-subclique. In fact, for any k > 3, this upper bound is achievable by a sparselyconnected three-parameter network. Proposition 1. There exists a family of threeparameter Hopfield networks with z = 1, y = 0 storing all k-cliques as bk/2c-stable states. The proof relies on the following lemma, which gives the precise condition for the three-parameter Hopfield network to store k-cliques as r-stable states for fixed r. Lemma 1. Fix k > 3 and 0 ≤ r < k. The Hopfield network (J(x, y), θ(z)) stores all k-cliques as r-stable states if and only if the parameters x, y, z ∈ R satisfy